IMM1P018 



CLAIMS 



1. A computer-implemented method for simulating the spatial interaction of a 
usfer-manipulated displayed first simulated object with a displayed second simulated object in 
a computer-simulated spatial environment, such that the user is provided with force feedback 
that represents the interaction between the first simulated object and second simulated object, 
the methVl comprising the steps of: 

a) generating within a computer-simulated spatial environment a first simulated object 
on a display dfevice, said first simulated object having a simulated location within said 
simulated spatial environment; 

~o) -simulating ihider computer control the motion of said first simulated object 4n 
response to motion of avphysical object of an interface device controlled by a user, wherein 
said physical object has\a physical position in a physical environment, and wherein a 
mapping between said simulated location of said first simulated object and said physical 
position of said physical object is created such that the physical position of the physical object 
in the physical environment has y&sphiaA correspondence to the location of said first 
simulated object in said simulates spatial environment; 

c) generating within said cem^ter-simulated spatial environment a second simulated 
object on said display device, said second simulated object defining a set of boundaries in 
said motion simulation and on said display device; and 

d) breaking said mapping between said simulated location of first simulated object and 
said physical position of said physical object when said first simulated object interacts with 
said second simulated object such that sai^physical position of said physical object is no 
longer spatially correlated with said simulatecMocation of said simulated object, said breaking 
occurring under conditions effective to provide, visual feedback and force feedback to said 
user which are effective to impart to said user\a physical sensation that corresponds to a 
realistic, simulated physical interaction of said i^rst simulated object with said second 
simulated object. 
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\ 2. The method of claim 1, wherein said physical position of said physical object is 
mapped to said simulated location of said first simulated object by a position control 
mapping. 

3. The method of claim 2, wherein said spatial correspondence includes a scale factor. 

4. TheVnethod of claim 1, wherein said simulated location of said first simulated 
object includes\a displayed location on said display device that is correlated with said 
simulated position^ 

5. The method, of claim 4, wherein said first simulated object interacts with said 
second simulated objec^after moving along a trajectory which intersects boundaries of said 
second simulated object. \ 

-6. The method ^ofxIaims4, wherein, said force .feedb.ackxQrrespmds..tO-axestoring force 
having a magnitude that is proportional to an amount that said mapping is broken. 

7. The method of claim 6, wherein said amount of said breaking of said mapping is 
proportional to the degree to which said physical position of said physical object exceeds the 
result of said interaction between said tksLsimulated object and said second simulated object 
as determined by said simulation. / \A \ / 

8. The method of claim 7, Wherein s\id restoring force is a spring force having the 
mathematical form: \ 

where F is said restoring force, x is a magnitude of a deviation of said spatial 
correlation including a deviation between the current location of the first simulated object and 
a location of said first simulated object had said mapping, not been broken, and £ is a spring 
constant parameter. \ 

9. The method of claim 8, wherein said restoring forceVicludes a damping force and 
said restoring force has the mathematical form: \ 

F = kx + bv \ 
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where F is said restoring force, x is a magnitude of a deviation of said spatial 
correspondence including a deviation between the current location of the first simulated 
object and a location of said first simulated object had said mapping not been broken, v is a 
function of a velocity of said physical object, and k and b are constant parameters. 

10. The method of claim 8, wherein said restoring force includes an intertial force and 
said restoringtorce has the mathematical form: 

F = kx +\y + ma 

where F is said\estoring force, x is a magnitude of a deviation of said spatial 
correspondence including^ deviation between the current location of the first simulated 
object and a location of said first simulated object had said mapping not been broken, v is a 
function of a velocity of said \hysical object, a is a function of an acceleration of said 
physical object, and k, b and m areSconstant parameters. 

11. The method of claim 10, wherein said inertial force corresponds to the movement 
of said second simulated object in response to said interaction between said second simulated 
object and said first simulated object. \ 

12. The method of claim 10, wMereiK ^IdNrestojang force includes a component 
resulting from friction between said simiflated/^qject and said simulated spatial environment. 

13. The method of claim 1, wherein said second simulated object can be independently 
moved within said simulation and on said display device. 

14. The method of claim 13, wherein said second\imulated object is controlled by 
said computer within said simulation. \ 

15. The method of claim 13, wherein user is a first user manipulating a first physical 
object, and wherein said second simulated object is controlled by a\econd user manipulating 
a second physical object of a second interface device. \ 

16. The method of claim 13, wherein said wherein said force feedback corresponds to 
a restoring force having a magnitude that is proportional to a degree that >said mapping is 
broken. \ \ 
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\ 17. The method of claim 16, wherein said restoring force is a spring force which is 
proportional to the degree to which the trajectories of said first and second simulated objects 
exceedihe result of said interaction between said simulated objects as determined by said 
simulation^ 

18. Thesmethod of claim 17, wherein said restoring force is a spring force having the 
mathematical fonn: 

F = k(x } +x 2 \ 

where F is saidVestoring force, x x is a magnitude of a deviation of said spatial 
correlation including a deviation between the current location of the first simulated object and 
a location of said first simulated object had said mapping not been broken, ^ is a magnitude 
of a deviation of said spatial correlation including a deviation between the current location of 
the second simulated object and V location of said second simulated object had said mapping 
not been broken, and A: is a spring constant parameter. 

19. The method of claim 18, wherein said restoring force includes a damping force 
having the mathematical form: \f \ 

F- k{x x +x 2 ) + b{v x + v 2 ) / \ / ^ — 

where F is said restoring force, v, is a function of the velocity of said first physical 
object, v 2 is a function* of the velocity of said second f)hysical object, and k and& are constant 
parameters. \ 

20. The method of claim 18. wherein said restoring ibrce is determined, at least in part, 
utilizing a weighting factor for influencing the location of\aid first and second simulated 
objects resulting from said interaction. \ 

21. The method of claim 20, wherein the location on said display, L of the simulated 
objects shown on said display is determined by the equation: \ 

(w!+w 2 ) 
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\22. The method of claim 16, wherein said first simulated object and said second 
simulated object are displayed at an equilibrium position determined, at least in part, by a 
position onsaid first physical object and a position of said second physical object. 

23. TheViethod of claim 22, wherein said equilibrium position is determined, at least 
in part, by a position of said first physical object, by a position of said second physical object, 
and by a weightingSfactor. 

24. The methoa\pf claim 16, further comprising a step of generating a plurality of 
simulated objects on saicl display device and within said simulation, wherein each of said 
plurality of simulated objects is controlled by one of a plurality of users each using an 
associated interface device. \ 

25. The method of claim V wherein said second simulated object is an obstruction 
object. \ 

26. The method of claim 25, therein a location of said obstruction object cannot 
change within said simulation. \ 

27. The method of claim 1, wherein said first simulated object is a paddle and said 
second simulated object is a ball object. \ /^fl ^/ 

28. The method of claim 1, wherein s^jdwmulated environment is a sporting 
environment including a playfield. \ 

29. A computer system for simulating the spatial interaction of a displayed first 
simulated object with a displayed second simulated object in a\computer-simulated spatial 
environment such that the user is provided with a force feedback that realistically represents 
said interaction, comprising: \ 

a processor for executing a simulation including a first simulated object, said 
simulation being configured to implement the motion of said first simulated object in 
response to motion of a physical object of an interface device controlled by V user, wherein 
said physical object has a physical position in a physical environment, and wherem a position 
control mapping between said simulated location of said first simulated object and said 
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physical position of said physical object is created, said simulation being further configured 
, to generate a second simulated object having boundaries such that said second simulated 
object impedes the simulated motion of said first simulated object when the trajectory of said 
first simulated object intersects said boundaries of said second simulated object; 

a ^display device for viewing the location and motion of said first simulated object and 
said second simulated object; and 

a force feedback mechanism configured to impart to a user of said computer system a 
physical sensanpn that corresponds to the simulated physical interaction of said first 
simulated object with said second simulated object when the trajectory of said first simulated 
object intersects the boundaries of said second simulated object, wherein said position control 
mapping between saidgphysical object and said first simulated object is broken during said 
interaction when providing said physical sensation. 

30. The computer system of claim 29, wherein said force feedback mechanism is 
configured to generate a restoring force that is proportional to the magnitude by which said 
mapping is broken when said first^ simulated object intersects said boundaries of said second 
simulated object. 

31. The computer system of clziirn 30, wherein said magnitude is proportional to the 
degree to which said commands, when processed using said simulation, exceed the result of 
said interaction between said first simulated object and said second simulated object as 
determined by said simulation. 

32. The computer system of claim 30, wherein said restoring force is a spring force 
having the mathematical form: 

F = kx 

where F is said restoring force, x is a magnitude d£ a deviation of said spatial 
correlation including a deviation between the current location of tne first simulated object and 
a location of said first simulated object had said mapping not been \roken, and A: is a spring 
constant parameter. 

33. The computer system of claim 32, wherein said restoring forc^includes a damping 
force and said restoring force has the mathematical form: 
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F = kx + bv 



Nwhere F is said restoring force, x is a magnitude of a deviation of said spatial 
correspondence including a deviation between the current location of the first simulated 
object andsa location of said first simulated object had said mapping not been broken, v is a 
function of awelocity of said physical object, and k and b are constant parameters. 

34. The method of claim 33, wherein said restoring force includes an intertial force 
corresponding to\the movement of said second simulated object in response to said 
interaction between\aid second simulated object and said first simulated object and said 
restoring force has the fnathematical form: 

F = kx + bv + mV 

where F is said restonW force, x is a magnitude of a deviation of said spatial 
correspondence including a deviation between the current location of the first simulated 
object and a location of said first simulated object had said mapping not been broken, v is a 
function of a velocity of said physical object, a is a function of an acceleration of said 
physical object, and k, b and m are constant parameters. 

35. The method of claim 34, wherain said restoring force includes a component 
resulting from friction between said simulated o£je9^aqd said simulated spatial environment. 



36. The computer system of claim 30, wtp 
moved within said simulation and on said display 



nn saki second simulated object can be 



Levi 



37. The computer system of claim 36, wherein saicrsecond simulated object moves on 
said display device during said simulation in response to manipulations of a second physical 
object of a second interface device by said second user, said\econd interface device being 
coupled to a second computer system coupled to said computef^system through a network 
interface. 



38. The computer system of claim 36, wherein said whereirk said force feedback 
corresponds to a restoring force that is proportional to the magnitude of saki breaking of said 
mapping. 
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39. The computer system of claim 38, wherein said restoring force is a spring force 
wmch is proportional to the degree to which the trajectories of said first and said second 
graphical player objects exceed the result of said interaction between said simulated objects as 
determined by said simulation. 

40. \The computer system of claim 38, wherein said restoring force includes a 
weighting factor such that the location L on said display device of the simulated objects 
shown on said ^display device is determined by the equation: 



(w, 



41. The computerssystem of claim 29, wherein said user interface device is operated 
manually, and said display device at least partially conceals the interface device from the view 
of said user such that said us^r cannot view the user's hands when operating said interface 
device. 

42. The computer system oXclaim 29, further comprising a second visual display 
device and a second user interface device, and said simulation is configured to generate 
location and kinematic information about said first simulated object in response to input 
supplied by both of said user interface devic 

43. The computer system of claim 29\yC r froi£if* x said processor is coupled with a 
second processor executing said simulatioiVstfid second processor being responsive to input 
from a second interface device, said processors being coupled such that said simulations 
communicate input information from said interface devices. 



44. An apparatus for allowing a user to naturally interact with a computer-generated 
spatial sporting simulation, the apparatus comprising: 

a processor for implementing a computer-generated spatial spirting simulation, said 
simulation including a first simulated object and a ball object for interacting with said first 
simulated object; 

a display device coupled to said processor for displaying said first simulated object; and 
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\ a force feedback interface device coupled to said processor and including a physical 
useV object which is grasped by a user, said user object having two degrees of freedom and 
beingXmovable in a planar workspace corresponding to a simulated plane within said 
simulation to move said first simulated object within said simulation according to a position 
control mapping, and wherein said processor is operative to command forces output to said 
user via saiaSforce feedback interface device. 

45. An apparatus as recited in claim 44 wherein said display device includes a display 
screen for displaying images thereupon. 

46. An apparatus as recited in claim 45 wherein a plurality of users operate said 
apparatus, and wherein\aid display device includes a plurality of display screens such that 
each user views an associated display screen. 

47. An .apparatus as Recited "in claim 44 wherein said display device Includes a 
projection device for displaying images upon a surface. 

48. An apparatus as recited in claim 47 wherein said projection device is a front- 
projection device positioned on the same side of said surface as said images displayed on 
said surface. \ /""^ . 



49. An apparatus as recited in claink47 ^herein said projection device is a rear- 
projection device positioned on the opposite side of said surface to said images displayed on 
said surface. \ 

50. An apparatus as recited in claim 44 wherein said force feedback interface device 
includes a first link and a second link, said first link Toeing rotatably coupled to ground and 
said second link being rotatably coupled to said first liW, and wherein said user object is 
coupled to said second link. \ 

51. An apparatus as recited in claim 50 wherein said r&rce feedback interface device 
includes a five member linkage. \ 

52. An apparatus as recited in claim 51 wherein said five member linkage includes a 
first link coupled to a ground, a second link coupled between said first link and said user 
object, a third link coupled to said ground, and a fourth link coupled between said third link 
and said user object. \ 
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53. An apparatus as recited in claim 50 wherein said user object can be moved in a 
rckary third degree of freedom, and wherein a displayed orientation of said first simulated 
object is changed according to movement of said user object in said third degree of freedom. 



S4. An apparatus as recited in claim 44 wherein said user object includes a racquet 



handle. 



55. Ais apparatus as recited in claim 54 wherein said interface device includes a slider 
coupled to said handle for moving said handle in a linear degree of freedom along a support, 
said support being rotatably coupled to a ground member for moving said handle in a rotary 
degree of freedoim 



56. A method fot providing force feedback for interacting simulated objects in a 
simulation implemented oka computer system, the method comprising the steps of: 

displaying a user-controlled first simulated object and a second simulated object on a 
display device of a computer system, said first simulated object moving on said display 
device during a simulation in response to manipulations of a physical object of an interface 
device by a user, said interface device being coupled to said computer system; 



determining when said first simr 
within said simulation; 



set engages said second simulated object 



displaying said determined engagetnen\of said first simulated object with said second 
simulated object, wherein said first simulated object has a predetermined simulated 
compliance and said second object has a predetermined simulated mass: and 

outputting a force command to said interface\device to apply a force to said physical 
object manipulated by said user in at least one degreeNpf freedom provided by said interface 
device, said force being applied in the direction of said engagement of said second simulated 
object with said first simulated object and having a magnitude in accordance with said 
simulated mass of said second simulated object. 

57. A method as recited in claim 56 wherein a simulated compliance of said first 
simulated object additionally affects said magnitude of said force.\ 
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5\. A method as recited in claim 57 wherein simulated velocities of said first 
simuIateaNpbject and said second simulated object additionally affect said magnitude of said 
force. 
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59. A method as recited in claim 57 wherein a simulated gravity acting on said 
simulated object^dditionally affects said magnitude of said force. 

60. A method^s recited in claim 56 wherein said first simulated object elongates at a 
point of impact with\^aid second simulated object in accordance with said simulated 
compliance of said first simulated object. 

61. A method as recited in claim 60 further comprising a step of detecting whether 
said user has input a command to trap said second object such that said second simulated 
object remains engaged with said, first simulated object, thereby allowing said user to move 
said first simulated object and said second simulated object. 

62. A method as recited in claim 61 wherein said command to trap is input by said 
user pressing a button of said interface^ device, wherein said ball is released from said 
engagement with said paddle when said buttpn is released. 



63. A method as recited in claim 61 
having two endpoints and said second sir 
and said ball object can be moved by saic 
paddle is a flexible sling. 



aid first simulated object is a paddle 
^bBiect is ajarfdl object, wherein said paddle 
§er Wtien said ball object is trapped as if said 



64. A method as recited in claim 63 wherein wften said trap command is removed 
while said ball object is moved by said user in an approximately circular path, said ball object 
is disengaged and moved tangential to said circular path awaytfrom said paddle. 

65. A method as recited in claim 63 wherein when said trap command is removed 
while said second simulated object is moving toward said two endpoints of said paddle, said 
ball object remains engaged with said paddle until said ball object is^moved past said two 
endpoints, and is then forced away from said paddle. \^ 

66. A method as recited in claim 63 wherein when said trap command is removed 
while said ball object is moving away from said two endpoints of said paddle, said ball 
object is disengaged from said paddle and is moved away from said paddle. 
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67. A method as recited in claim 61 wherein when said command to trap is received, 
lid ball object and said paddle are held in a configuration that was present when said 
command to trap was received, and wherein forces associated with said ball object interacting 
with said paddle are turned off and said paddle is modeled as a rigid object. 




68. A method as recited in claim 67 wherein when said command to trap is removed, 
a reaction for^e is provided to said physical object which is opposite in direction from the 
force that woulaN^ave been provided by said simulation in the absence of said command to 
trap. 

69. A method as\ecited in claim 61 wherein when said command to trap is received, 
forces associated with sar<J ball object interacting with said paddle remain active and said 
paddle remains compliant. 

70. A method as recited Ik claim 61 further comprising a step of displaying a third 
simulated object in said simulation ,\aid third simulated object being controlled by a second 
user by a second physical object ancNbeing operative to trap said second graphical object 
when said second user inputs a commands to trap said second simulated object, and wherein 
said first and third simulated objects can concurrentiy^rap said second simulated object, such 
that force feedback based on movement of said^st slhnulatejf object is output to said second 
physical object and force feedback based on nWement of said third simulated object is 
output to said first physical object when both said simulated objects trap said ball. 

71. A method as recited in claim 56 further comprising displaying a simulated goal 
object on said display screen, wherein said user moves said first simulated object to block 
said second simulated object from moving into said simulate&seoal object. 

72. A method as recited in claim 56 further comprising a displaying a third simulated 
object on said display device, said third simulated object moving, on said display device 
during a simulation in response to manipulations of a second physital object of a second 
interface device by a second user. \ 

\ 

73. A method as recited in claim 72 wherein said second interface device is coupled to 
said computer system. 
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i^4. A method as recited in claim 72 wherein said computer system is a first computer 
system, \and wherein said second interface device is coupled to a second computer system 
that is linked with said first computer system. 

75. Ah, interface system for allowing a player to naturally interact with a computer- 
generated simulation, said force feedback interface device comprising: 

a processor fordmplementing a computer-generated simulated environment including a 
computer-generated player object and a computer-generated paddle object associated with 
said computer-generated pi^yer object; 

a display device coupled\to said processor for displaying said player object and said 
paddle object; \^ 

..a physical -object ..to -.be _grasp.eJL.by ~a ..player .and incJ udi ng jtrans ducexs jsjich that , the 
motion of said physical object is communicated to said processor and such that forces can be 
commanded on said physical object by said processor, wherein said player manipulates the 
physical object to control a location of saicypaddle object within said computer-generated 
simulated environment according to a positioii-cbptrol mapping; and 

a platform supporting said player and including a sensor that is coupled to said 
processor, wherein said player manipulates the playens body relative to said sensor of said 
stage to control a location of said computer-generated player object within said computer- 
generated simulated environment according to a rate controkmapping. 

76. An interface system as described in claim 75^vherein said forces can be 
commanded on said physical object in two degrees of freedom. 
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